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® Method and apparatus for generating size and orientation Invariant shape features. 

@ A system extracts a set of size invariant, rotation invariant features from pixel data of a character and sends 
the set of features to a statistical decision tree to effectuate automatic recognition of a character. The set of 
extracted features includes a first group obtained by generating arrays representable as dimensions and other 
geometric qualities of six minimum bounding rectangles rotated atjout the character, including the distances 
between the center points of the rotated bounding rectangles. A second group of features is extracted by 
generating a group of arrays representing the perimeter pixel locations, radii lengths, and direction codes. The 
arrays of that group are "resampled" to generate corresponding 64 point arrays. The resampled an-ays of 
direction codes is smoothed and incremental direction codes are computed and compared to preselected 
thresholds to obtain features that indicate prominent convex and concave portions of the character. A third group 
of features is extracted by dividing a circle boundinig the character into eight rings and 24 slices and counting 
the number of character pixels contained in each ring and in each slice. Various Fourier transforms, autocor- 
relations, moment calculations, and sorting operations are performed on many of the aforementioned an-ays to 
yield further size and orientation invariant features. The distance between the centroid of the largest hole of the 
character and the center of the character is computed. The ratios of the perimeters of first and second holes of 
the character to the outer perimeter are computed. 
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METHOD AND APPARATUS FOR GENERATING SIZE AND ORIENTATION INVARIANT SHAPE FEATURES 



BACKGROUND OF THE INVENTION 

The invention relates to methods and apparatus for recognizing unconnected characters, such as 
alphanumeric characters and the like, and more particularly to methods and apparatus for efficient 
extraction of features which can be more efficiently utilized by statistical decision trees to recognize 
electronically scanned characters, and particularly to such methods and apparatus capable of produc- .g 
features which are generally size invariant and rotation invariant. 

In the past there have been two commonly used approaches to recognition of disconnected characters, 
one approach being a "structural" approach and the other being a "statistical" approach. In the structural 
approach, a character is skeletonized by means of a medical axis transformation, well-known to those 
skilled in the art. and then parts of the character are identified through a spacial" analysis of the skeleton. A 
wide variety of techniques for identifying the parts of the character are known to those skilled in the art. 
including analysis by means of procedural rule bases, analysis by means of one-dimensional and two- 
dimensional grammars, and also by means of structural decision tree analysis. The medial axis transform? 
tion is computationally very expensive. The computer processing required by the structural methodolog 
gives rise to the identification of meaningless "false" or "noise" structures of the scanned character. In 
processing such noise structures, which may be very numerous for a single character, complex recognition 
algorithms are necessary to avoid misclassification. . , ^ . 

The statistical approach to character recognition involves extraction of "features" from the pixel data 
obtained by scanning the character and feeding the extracted features into a statistical decision tree which 
compares them to a preselected set of features for various predefined character classes and eventually 
recognizes or rejects the character. Prior character extraction techniques have been confined mostly to 
mass sampling within a rectangular grid, generation of two-dimensional moments. Founer transforms of 
certain boundary properties, the aspect ratio of the character, the thinness ratio of the perimeter length 
versus the number of dark pixels in the character, and the like. Prior statistical techniques for operatng ori 
extracted features generally require that the size and orientation of the characters be known. Additional 
adequate statistical feature extraction techniques have been devised but they generally require large 
amounts of computer processing time and memory capacity. 

Thus there is an unmet need in the character recognition art for an improved character feature 
extractiorl method and apparatus that produces an efficient set of size invariant;- rotation invariant features 
that can be processed by state-of-the-art decision trees, such as those in accordance with "ISOETRP-An 
Interactive Clustering Algorithm with New Objectives", by C. Y. Suen and Q. R. Wang, in Pattern 
Recognition. Vol. 17. No. 2. pp. 211-219. 1984. to allow efficient statistical character recognition to 
rapidly accomplished by one or more state-of-the-art microprocessors, such as the Motorola MC68020 
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microprocessor. 



SUMMARY OF THE INVENTION 



It is an object of the invention to provide a system for extracting a set of features all of which are 
independent of size and orientation of the character and which are efficient in recognizing the character by 
means of a statistical decision tree. . . 

It is another object of the invention to provide an method and apparatus for rapidly recognizing 
45 characters using computer hardware that is much less complex than has been used in the prior art. 

It is another object of the invention to generate a "robust" group of features from scanned character 
data wherein the features have probability distributions with differentiated means and small standard 
deviations. 

It is another object of the invention to extract a "rich" group of features from scanned character data 
50 wherein the features are minimally correlated so as to provide substantially different information about the 
shape of the character. 

Briefly described, and in accordance with one embodiment thereof, the invention provides a system for 
effectuating extraction of features of an electronically scanned character, which features are size indepen- 
dent and rotation independent, to be supplied to a statistical decision tree to effectuate recognition of the 
character. In the described embodiment, one set of features is extracted by determining a plurality of 
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predetermined parameters of a fixed number (e.g., 6) of hypothetical minimum bounding rectangles each of 
which has four sides that touch four respective points of the character, each such minimum bounding 
rectangle being rotated relative to to others by a predetermined angle (e.g.. 15 degrees). One-dimensionai 
arrays are generated containing, respectively, the height and width of each minimum bounding rectangle. 

5 the area of each minimum bounding rectangle, the aspect ratio of each minimum bounding rectangle, the 
center point of each minimum bounding rectangle, and the distances between the center points of each 
minimum bounding rectangle. A feature is obtained by computing the length of the perimeter of the polygon 
defined by the center points of the minimum bounding rectangles. An an^ay of x coordinates of each 
respective perimeter pixel of the character is generated, and a similar array of y coordinates of each 

TO perimeter pixel is generated. An array is generated containing the lengths of radii from the center point of 
the character to each perimeter pixel thereof. An anray of direction codes, each having a value of 0 through 
7. is generated for each pair of perimeter pixels of the character. Another array is generated containing the 
distance from each perimeter pixel to the next, each pixel distance being 1 if the next pixel is horizontally or 
vertically aligned with the present perimeter pixel, and is equal to the square root of 2 if the next perimeter 

TS pixel is diagonally aligned with the present pixel. The arrays of x and y coordinates of the perimeter pixels 
are resampled so that each includes a fixed number (e.g.. 64) of resampled x or y coordinates, each of 
which is an average of an equal fraction of the total number of x or y coordinates, respectively. Two 
complex Fourier transforms are performed on the resampled x and y coordinates to produce two 
corresponding groups of eight magnitude-squared harmonic coefficients, each of which is normalized by 

20 dividing it by the sum of the eight con-esponding harmonic coefficients, and then taking the square roots. 
The array of lengths of radii is resampled by operating on the anray of pixel distances to produce a fixed 
number (e.g., 64) of resampled lengths of radii, each representing an equal proportion of the total perimeter 
length of the character. The array of direction codes is resampled by operating on it and the anray of pixel 
distances to produce a fixed number (e.g.. 64) of resampled direction codes each representing the 

25 perimeter tangent angle over equal proportions of the total perimeter length of the character. An increment 
of circularity is subtracted from each resampled direction code to produce an anray of con-ected resampled 
direction codes. 

A moving average of the resampled direction codes is operated upon to generate a fixed number (e.g.. 
64) of direction code increments each of which is equal to the difference between one moving average 
30 number and the next. Each of the direction code increments is compared to a threshold, and a concavity 
indicator number is incremented if that direction code increment is less than the threshold, and a convexity 
indicator number is incremented if that direction code increment is greater than the threshold. Multiple 
thresholds can be specified to measure different extremes of concavity and convexity. 

A fixed size an-ay (e.g., 8) of ring variables are assigned to con-esponding hypothetical contiguous ring 
35 regions of equal radius increments in a circle bounding the character. The x and y coordinates of each pixel 
in the character are systematically incremented to determine if the corresponding pixel is dark, and if this is 
the case, it then is determined in which of the ring regions the present pixel is located, and its 
con-esponding ring variable is incremented. After all dark pixels of the character have been thereby 
effectively associated with a ring region, each of the ring variables is scaled by dividing it by the total 
40 number of dark pixels in the character. 

A fixed size array (e.g., 24) of slice variables are assigned to con-esponding hypothetical contiguous 
pie-shaped slice regions (e.g., subtending 15 degrees) of a bounding circle of the character. The x and y 
coordinates of each pixel in the character and systematically incremented to determine if the corresponding 
pixel is dark, and if this is the case, it then is determined in which of the slice regions the present pixel is 
46 located, and the corresponding slice variable is incremented. After all dark pixels of the character have been 
thereby effectively associated with a slice region, each of the slice variables is scaled by dividing it by the 
total number of dark pixels in the character. A real Fourier transform operation is separately performed on 
the bounding rectangle arrays, resampled perimeter an-ays of radii lengths and corrected resampled 
direction codes, and the slice array to produce conresponding groups of eight magnitude-squared hamnonic 
50 coefficients, each of which is normalized by dividing it by the sum of the eight harmonic coefficients, and 
taking their square roots. Normalized autocorrelation operations are performed on these same arrays. 

The first four one-dimensional moments, namely the average, variance, skew, and kurtosis, are 
separately calculated for the length, area, aspect ratio, and center point distance bounding rectangle arrays, 
the resampled perimeter arrays of lengths of radii and direction code increments, and the ring and slice 
55 arrays. Sorting operations are performed on these same arrays, and various arithmetic combinations of sets 
of elements obtained therefrom are computed. Moments and sorted elements are suitably scaled by 
precomputed length or area variables where appropriate. 

The scaled distance between the centroid of the largest hole in the character and the average of the 
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centers of the minimum bounding rectangles of the character is computed. The ratio between the perimeter 
of the two largest holes of the character and the perimeter length of the character also are computed. 

The set of extracted features includes the sets of eight normalized harmonic coefficients and normal- 
ized autocorrelations of selected bounding rectangle arrays, perimeter arrays, and the slice array, the four 
5 one-dimensional scaled moments and the scaled sorted elements and their various arithmetic combinations 
of selected bounding rectangle arrays, perimeter arrays, ring and slice arrays, the concavity and convexity 
indicators, the scaled sum of the distances between adjacent centers of the minimum bounding rectangles, 
the ratios involving the perimeters of holes of the characters, and the scaled distance between the centroid 
of the largest hole of the character and the center of the character. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Rg. 1 is a generalized flow chart of the feature extraction technique of the present invention. 
7S Fig. 1A is a flow chart of the subroutines called by the flow chart in Rg. 1 to compute various 

features. 

Rg. 2 is a diagram useful in explaining various input variables utilized by the feature extraction 
process of Rg. 1 . 

Rg. 2A is a diagram useful in explaining minimum bounding rectangles used in the present inventiov 
20 Fig. 2B is a diagram useful in explaining the lengths of radii and perimeter distances used in the 

present invention. 

Rg. 20 is a diagram useful in explaining ring and slice integration used in the present invention. 
Fig. 2D is a diagram useful in explaining the features hdist hrati , and hrat2 computed in the present 
- invention. 

25 Rgs. 3-6 constitute a flow chart of the DOCON subroutine of Rg. 1 . 

Fig. 7 is a flow chart of the DOPERIM subroutine of Rg.1. 
Rg. 8 is a flow chart of the FSAMP subroutine of Rg. 1 . 

Rg. 8A Is a diagram useful in understanding the operation of the flow chart of Rg. 8. 

Rgs. 9 and 10 constitute a flow chart of the CDFOURIER subroutine of Fig. 1. 
30 Rg. 11 is a flow chart of the PFSAMP subroutine of Rg. 1. 

Rg. 11 A is a diagram useful in understanding the operation of the flow chart of Fig. 11. 

Rg. 1 1 B is a diagram also useful in understanding the operation of the flow chart of Fig. 1 1 . 

Rg. 12 is a flow chart of the DERIV subroutine of Rg. 1. 

Rg. 13 is a flow chart of the HILOSUM subroutine of Rg. 1. 
35 Rg. 14 is a diagram useful in understanding the operation of the subroutine of Fig. 13. 

Rg. 15 is a flow chart of the DORAD subroutine of Fig. 1. 

Rg. 18 is a flow chart of the RFOURIER subroutine of Fig. 1, 

Rg. 17 is a flow chart of CORREL subroutine of Fig. 1. 

Fig. 18 is a flow chart of the MOMENT subroutine of Rg. 1A, 
40 Rg. 19 Is a flow chart of the SORT subroutine of Rg. 1A. 

Fig. 20 Is a flow chart of the DOHOLE subroutine of Rg. 1 . 

Rg. 21 is a diagram of a document recognition system In which the feature extraction program of Rg. 
1 is embodied. 



DESCRIPTION OF THE INVENTION 

At the outset, it should be understood that the feature extraction technique of the present Invention 
receives a group of input variables, listed in Table 1, from a file that has been produced in response to line- 
so by-line scanning of a document containing the characters to be recognized. Those skilled in the art are 
familiar with a variety of well-known border tracking algorithms that produce various input variables which 
are usable in a character recognition system by operating on pixel data produced by scanning the object. In 
the presently preferred embodiment of the invention, a conventional border scanning subroutine operates on 
an "object" or object file. 

55 That object includes all of the pixel data of the character to be recognized, assembled as horizontal 
slices that correspond to the geometric structure of the character on the document scanned. 

Table 1 shown directly below is a list of the input variables produced by the above border tracking 
routine and definitions of those variables. 
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TABLE 1 
Input Variables 

p(x.y) binary pixel image (0 « light / 1 « dark) , where 

all dark pixels are connected 

X X coordinate of P(x,y): 0 x <« mx 

y y coordinate of P(x,y): 0 <« y <« mx 

mx maximum x coordinate of P<x,y) «= 1 

my maximum y coordinate of P(x,y) « 1 

npix number of dark pixels P(x,y) =1 

PX(ip) counter-clockwise traced from upper leftmost 

P(x,y) « 1 perimeter pixel x coordinates 

PYCip) counter-clockwise traced from upper leftmost 

P{x,y) « 1 perimeter pixel y coordinates 

PD(ip) 8-way directions (0 - 7) from perimeter pixel 

ip to ip+1 (or 0 if last pixel) 

ip perimeter pixel index: 0 <= ip <= nperim - 1 

nperim number of perimeter pixels 

perim perimeter length (adding 1 for each horizontal 

or vertical (even) direction, and 2 for each 
diagonal (odd) direction)* 

HX(ih) counter-clockwise traced from upper leftmost 

P(x,y) « 0 largest hole border pixel x 
coordinates 

Hy(ili) coxinter-clockwise traced from upper leftmost 

P(x,y) « 0 largest hole border pixel y 
coordinates 

HD(ih) 8-way directions (0 - 7) from largest hole 

border pixel ih to ih+1 (or 0 if last pixel) 

ih largest hole border pixel index: 0 <= ih <= nhole 

j^ole number of largest hole border pixels (0 if none) 

ahole largest nhole border length (0 if none) 

bhole second largest hole border length (0 if none) 
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Rg. 2 is a diagram of a character to be recognized, having the shape of a "B". It will be convenient to 
refer to Rg. 2 in describing the input variables of Table 1 . In Table 1 P(x,y) is a two-dinnensional array, and 
represents the binary pixel image. If P(x,y) is a at the present x.y coordinate, it represents a dark pixel; 

5 similarly, if P(x.y) is a "0". it represents a light or transparent pixel at the coordinate x.y. As indicated along 
X axis 11 in Rg. 2, x can have any value between 0 and mx. Similarly, as indicated by y axis 12 in Rg. 2. y 
can have any value between 0 and my. The minimum bounding rectangle at 0 rotation is defined by the 
points 0.0. O.mx. mx.my and O.my as shown. The total number of dark pixels in character 10. i.e., the 
number of pixels for which P(x,y) equals "1 is equal to npix. 

70 PX(ip) is a one-dimensional array of the x coordintes of the nperim perimeter or pixels of character 10. 
The index variable ip can have any value between 0 and nperim-1, where nperim is the total numb^-ir of 
perimeter pixels of character 1 0. Similarly. PY{ip) is a one-dimensional array of all of the y coordinr^tes of 
the nperim perimeter pixels of character 10. The PX(ip) and PY(ip) arrays are listed in order of perimeter 
points traced counter-clockwise from the upper. leftmost pixel of character 10. that is. from point 13 in Rg. 

75 2. If character 10 is preprocessed by a program that, for example, "smooths". /thins", or "decomposes" 
the character, then the PX(ip) and PY(ip) arrays would represent the perimeter(s) of the result(s). 

The one-dimensional array PD(ip) represents one of eight directions from a perimeter pixel ip to the 
next perimeter pixel ip + 1 as the boundary of the character is traced counter-clockwise. PO(ip) can take any 
value from 0 through 7. as indicated in 9 of Fig. 2A. in which the eight directions representing 0 degrees. ^ 

20 degrees. 90 degrees, 135 degrees. 180 degrees. 270 degrees, and 315 degrees are represented by the 
codes 0. 1.2, 3. 4. 5. 8. and 7. respectively. Thus, the group of three arrays PX(ip), PY(ip). and PD(ip) 
represent all of the perimeter points and directions between consecutive pixels for the character 10. (Note 
that the PD(ip) array could be easily derived from P(x.y) or PX(ip). and PY(ip).) 

The variable perim represents the total perimeter length for the character 10. This length is determined 

25 by summing all of the distances between consecutive pixels as the perimeter of character 10 is traced 
counter-clockwise from pixel 13 back to pixel 17. For each even-numbered value- of PD(ip). the length of the 
horizontal or vertical distance, i.e.. 1 is added to perim. and for each odd-numbered value of POfip), the 
length of the diagonal distance (such as 16 or 19 in Rg. 2), i.e.. the square root of 2. is added to perim. in 
order to obtain the total perimeter length of character 10. 

30 HX(ih), HY(ih). and HD(ih) are the x coordinate, y coordinate, and direction code arrays for the largest 
hole border pixels, and are similar to PX(ip). PY(ip). and PO(ip). except that the former contain hole border x 
and y coordinates, and direction codes, respectively, if the character has a hole in it. ih is the largest hole 
border pixel index and can assume any value between 0 and nhole-1. where nhole is the total number of 
border pixels in the largest hole 15 of character 10. In Rg. 2. reference numerals 14 and 15 represent 

35 holes. Point 19 represents the starting point for tracing the boundary of hole 15 counter-clockwise back to 
point 20. As shown in Rg. 2. P(x.y) is 1 everywhere the character is dark, in locations of holes in the 
character, such as holes 14 and 15. P(x,y) is 0. Ught pixels are everywhere indicated by P(x,y) = 0 in Rr 
2. 

The length of the largest hole border, i.e.. the length of the boundary of hole 15. is represented by the 
40 variable "ahole". and has a value 0 if there is no hole in the character. The variable "bhole" is the border 
length of the second largest hole. i.e.. of hole 13 in character 10 if there is a second hole. If there is no 
second hole, bhole has a value of 0. 

Before describing the feature extraction program of the present invention, which is described with 
reference to the flow chart of Rg. 1 and the various subroutines called thereby, it will be helpful to first 
45 describe in some detail the computed variables, which are listed in Table 2, set forth below. 
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TXBLE 2 
Computed Variables 

clen(O-ll) lengths of sides of the bounding rectangles 

of P(x,y) rotated every 15 degrees 

cprd(0-5) areas of the bounding rectangles of P(x,y) 

rotated every 15 degrees 

cdiv{0-5) aspect ratiois of the bounding rectangles 

of P{x,y) rotated every 15 degrees 

cxcp(0-5) X coordinates of the center point of the 

bounding rectangle of P(Xry) rotated every 
15 degrees 

cycp(0-5) y coordinates of the center point of the 

bounding rectangle of P(x,y) rotated every 
15 degrees 

cx average x coordinate of the bounding 

rectangle center points 

taverage y coordinate of the bounding 
rectangle center points 

cdist(0-14) distances between the center points of 

every bounding rectangle 

cdsum s\m of the distances between adjacent 

bounding rectangle cehter points scaled 
by perim 

xproj(ip) array of nperim x coordinates of the 

perimeter pixels (same as PX(ip)) 

yproj(ip) array of nperim y coordinates of the 

perimeter pixels (same as PY(ip)) 

plen(ip) array of nperim distances between 

adjacent perimeter pixels (1 if PD(ip) even / 
V5"if PD(ip) odd) 

prad{ip) array of nperim distances from (cx,cy) to 

each (PX{ip) ,PY(ip) ) 

ptan(ip) array of nperim directions between adjacent 

perimeter pixels (same as PD{ip)) 
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prmax 

xsainp{0-63) 

ysamp (0-63) 

rsamp (0-63) 
ts amp (0-63 ) 

tnorm (0-63) 
tdif f (0-63) 

ccoef (1-8) 

dcoef (1-8) 

thisxim 

tlosum 

ring (0-7) 

slice(0-23) 

hx 
hy 

hdist 

hratl 
hrat2 



jnaxiiniiin distance from (cx,cy) to (PX(ip), 
PY(ip)0 



xproj 



yproj 



ptan 



resampled x projection of perimeter from 

i 

resampled y projection of perimeter from 

i 

resampled perimeter radii from plen and prad 

resampled perimeter tangents from plen and 

normalized perimeter tangents from tsamp 



smooth differences of perimeter tangents 
from tsamp 

first 8 complex Fourier coefficients 
(method 1) from xsamp'and ysamp scaled by 
sum of squares 

first 8 complex Fourier coefficients 
(method 2) from xsamp and ysamp scaled by 
sum of squares 

sum of perimeter tangent differences above 
given threshold from tdiff 

sum of perimeter tangent differences below 
given threshold from tdiff 

concentric rings of equal radius increment 
from radially sampled P(x,y) scaled by npix 

adjacent 15 degree slices of equal area from 
radially sampled P(x,y) scaled by npix 

x coordinate of largest hole border centroid 

y coordinate of largest hole border centroid 

ratio of distance between (hx,hy) and (cx^cy) 
to prmax 

ratio of ahole to perim 

ratio of bhole to perim 



In Table 2 some of the computed variables are merely intermediate variables, while others are the 
55 desired extracted features of the character to be recognized, and are supplied as inputs to a suitable 
statistical decision tree, such as one of the type described in the above-referenced Suen articles. 

Before describing the computed variables of Table 2, it will be helpful to refer to Rg. 2A, in which 
reference numeral 25 again designates a hypothetical character to be recognized. Numeral 32-0 represents 
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a first minimum bounding rectangle of character 25. In accordance with the present invention, a second 
minimum bounding rectangle 32-1 can also be drawn, rotated 15 degrees relative to the original x and y 
axes. Four further minimum bounding rectangles (not shown), each rotated 15 degrees clockwise (or 
counter-clockwise) relative to the next, also can be drawn. Each of the six minimum bounding rectangles 

5 may have a different center and a different height and a different width. clen(O) represents the height of the 
first minimum bounding rectangle 32-0, clen(7) is the width of that minimum bounding rectangle, cien(2) is 
the height of the first rotated minimum bounding rectangle 32-1. clen(8) is the width of rectangle 32-1, etc.. 
so that a set of variables clen(0),...clen(1l) represents the lengths of two perpendicular sides of all six 
rotated minimum bounding rectangles of P(x,y). 

10 cprd(O). cprd(1)....cprd(5) are the areas of the six minimum bounding rectangles of P(x,y). cdiv(0)....cdiv- 
(5) are aspect ratios of each of the six minimum bounding rectangles, each equal to the shorter cien value 
of that rectangle divided by its longer clen value, so that all values of cdiv(0-5) are less than or equal to 1 . 

The variables cxcp(0)....cxcp(5) are the x coordinates of the six center points of the respective minimum 
bounding rectangles. cycp(0),...cycp(5) are the y coordinates of the six center points of the respective 

75 minimum bounding rectangles, cx is the average x coordinate of the six minimum bounding rectangle 
center points, and cy is the average y coordinate of the six minimum bounding rectangle center points, 
cx.cy is used herein as the center point or center of the character. The 15 variables cdist(0),...cdist(l4) 
represent the 15 possible distances between the six minimum bounding rectangle center points, cdsum is 
equal to the sum of the distances between the six adjacent (with respect to rotation angle) minimum 

20 bounding rectangle center points, which is the length of the perimeter of the polygon indicated by arrow 
25A of Fig. 2A, defined by the center points 26-0 26-1 . etc. 

The array xproj(ip) is set to PX(ip) (as will be explained with reference to Rg. 7). Similarly, the yproj(ip) 
array is set to PY(ip) in Rg. 7. Thus, xproj(ip) and yproj(ip) are arrays of x and y coordinates, respectively. 
- of the perimeter pixels taken in counter-clockwise order from the upper leftmost pixel of the character, ptan- 

25 (ip) is a direction code array that is set to PD(ip) in Rg. 7, and thus is an array of the directions between 
adjacent perimeter pixels. 

The prad(ip) array of distances from each of the nperim perimeter points of the character 10 from the 
character center (also identified by numeral 26 in Rg. 2B) to that perimeter point. For example, the 
distances prad(O), prad(1). and prad(2) are designated by arrows 34-0. 34-1, and 34-2 in Rg. 2C. The 

30 corresponding perimeter points are designated by points 33-0. 33-1, 33-2 in Rg. 2B. xproj(O), xproj(l), and 
xproj(2) are the x coordinates of perimeter points 33-0. 33-1. and 33-2. respectively. yproj(O), yproj(l), and 
yproj(2) are the y coordinates of those same three perimeter points, prmax is the maximum length of any of 
the radii 34-0. 34-1. etc., of the character 25 taken around its entire perimeter, i.e., is the maximum value of 
prad(ip). plen(ip) is an array of the distances of each of the nperim perimeter points 33-0. 33-1, ...33 to the 

35 next. This distance is either 1 or the square root of 2. depending upon whether the corresponding PD(ip) is 
even or odd, respectively. The remaining computed variables in this section of Table 2 can be best 
understood by reference to the remaining flow charts. The resampling techniques of the present invention 
are best understood with reference to the flow charts that outline the resampling process. The Fourier 
coefficients are best understood by reference to the flow charts which show those formulas. 

40 The computed variables ring(0-7) represent one of eight concentric rings each having the same radius 
increment. This can be understood by reference to Rg. 2C, in which reference numeral 25 again designates 
a hypothetical character to be recognized. As in Rg. 2. the x axis and y axis are indicated by numerals 1 1 
and 12, respectively. The character center is designated by reference numeral 26. The maximum of the 
radii drawn from the center point 26 to each perimeter pixel, prmax, is indicated by arrow 31. Eight 

45 concentric circles 27-0. 27-1 ....27-7 are drawn around center 26 to form the eight rings. The value of each 
ring variable is equal to the number of dark pixels of the character P(x,y) about which that ring is drawn 
divided by npix. the total number of pixels in the character. 

Twenty-four pie-shaped slices, each subtending a 15 degree angle, are designated by reference 
numerals 30-0, 30-1 ....30-23. The first slice 30-0 is defined by radius 29-1, which makes a -7.5 degree angle 

50 with respect to a horizontal axis 28 extending through center point 26 and radius 29-1, which makes a +7.5 
degree angle with respect to axis 28. Numerals 29-2 and 29-23 bound other slices as shown in Fig. 2C. The 
value of each slice variable is equal to the number of dark pixels of the character P(x,y) about which that 
slice is drawn divided by npix. 

The variables hx, hy. hdist, hratl. and hrat2 can be understood with reference to Rg. 2D. Referring now 

55 to Rg. 2D, character 25 in the shape of an "8" is shown having a large hole 35 and a small hole 36. 
(English alphanumeric characters fall into three classes, those with no holes, those with one hole and those 
with two holes.) The point hx,hy designated by numeral 37 is the centroid of the largest hole 35 with 
respect to its boundary. As before, the center of character 25 is point cx,cy,. designated by numeral 26. 
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hdist is the distance 38 between largest iiole centroid 37 and character center 26. indicated by line 38 in 
Rg. 2D divided by the maximum radius prmax, indicated by line 39. hrat1 is the ratio of the largest hole 35 
bounding border length ahole to the character perimeter perim. hrat2 is the ratio of the bounding hole 
border length of smaller hole 36 to the character perimeter length perim. The values of.hrati and hrat2 also 
5 are very useful in distinguishing between "noise holes" that might appear in a hand-drawn character like a 
"K". and provide a way of discriminating between such noise holes and intended holes like the ones in the 
"8" of Rg. 2D. 

Before describing the feature extraction program in detail, it will be helpful to first refer to Rg. 21. which 
is a diagram illustrating the structure of the document scanning system in which the feature extraction 
70 system of the present invention is incorporated. In Fig. 21. a scanner 2 scans a hand-drawn document 1. 
producing pixels which are filtered by noise filters 2A, the output of which is fed into a runlength encoder 
2B. The runlength encoder 28 produces raw runlengths that are "built" into "objects" consisting. of raw 
runlengths or horizontal slices arranged in a manner corresponding to the configuration of objects scanned 
on drawing 1 . - 

75 An object classifier 38 determines from size and other geometric properties whether an object is small 
enough to be classified as a character, and if it is. feeds raw runlengths of the object into a runlength 
decoder 4A that converts the object runlengths back into the pixel image, i.e., to P(x,y) and computes 
mx,my and npix. A border tracker 48 then operates upon P(x,y) to produce the remaining input variables 
shown in Table 1 . 

20 The feature extraction system of the present invention is designated by reference numeral 5 in Fig. 21 . 
producing intermediate computed variables and extracted features included in Table 2 and indicated in 
Rgs. 1 and 1 A. The extracted features are fed into a decision tree classifier 6A, the output of which is fed 
through a rule based character context routine 68 and from there into a formatter 7, the output of which is 
loaded for editing into a workstation 8 including an IBM PC-AT computer, a keyboard, a high resolution 

25 graphics monitor, a high resolution plotter, a hard disk, and other suitable hardware. 

All of the blocks shown in Rg. 21 except drawing 1. scanner 2. and workstation 8 are included in a 
graphics processor 8A that includes a number of 68020 microprocessors and suitable memory. 

Referring next to Rg. 1, names of subroutines are contained within blocks, and are represented by 
capital letters. When a computed array represented by small letters is followed by *. the RFOURIER 

30 subroutine of Fig. 16 is executed to compute a set of eight normalized harmonic coefficients of that array, 
and the CORREL subroutine of Rg. 17 is executed to compute a set of n normalized autocorrelations of that 
array with n elements. When a computed array of variables represented by small letters is followed by + . 
the fvlOMENT subroutine of Rg. 18 is executed to compute the average, variance, skew, and kurtosis of that 
array, as indicated in Rg. 1A and the SORT subroutine of Fig. 19 is executed to. perform a predetermined 

35 sorting of that array and various arithmetic combinations of the sorted elements. In Fig. 1 , any computed 
variables or arrays of variables that are underscored in either Fig, 1 or Rg. 1A are extracted features, which 
are supplied to the above-mentioned statistical decision tree which performs the character recognition 
function. Moment and sort variables are subject to division by scaling factors, as subsequently explained, 
insure size invariance. 

40 Referring to Rg. 1, the input variables PX(ip). PY(ip). PD(ip). nperim. and perim are input variables to 
the DOCON subroutine 11. The DOCON subroutine effectively draws six minimum bounding rectangles 
around a character such as character 25 in Fig. 28. each minimum bounding rectangle being rotated 15 
degrees relative to the previous rectangle so that there are six minimum bounding rectangles oriented at 0 
degrees. 15 degrees. 30 degrees. 45 degrees, 60 degrees, and 75 degrees. 

45 The output variables clen(O-ll), cprd(0-5), and cdiv(0-5) are produced by the DOCON subroutine of 
Figs. 3-6. The real Fourier transform subroutine of Rg. 1 6 operates on the cien, cprd, and cdiv arrays to 
produce the clcoef(1-4), cpcoef(1-2). and cdcoef(1-2) normalized harmonic coefficients, which are extracted 
features. Also, the autocorrelation subroutine of Rg. 17 operates on these same arrays to produce the clcor- 
(1-11), cpcor(1-5). and cdcor(1-5) normalized autocorrelations which are extracted features. 

50 The DOCON subroutine of block 11 also computes the cdist(0-14). cx and cy variables, and the cdsum 
feature. Also, the clen. cprd. cdiv. and cdist arrays are operated upon by the MOMENT and SORT 
subroutines, the outputs of which then are scaled to produce extracted features as indicated in Fig. 1 A. 

The input variables PX(ip). PY(ip). PD(ip). nperim. perim. and the computed variables cx and cy are 
operated on by the DOPERIM subroutine of Rg. 7 to perform an analysis of points around the perimeter of 

55 the character from which features are to be extracted. The DOPERIM subroutine 12 computes the xproj(ip). 
yproj(ip), prad(ip), ptan(ip), and pien(ip) arrays. DOPERIM also computes the prmax variable used by 
DORAD and DOHOLE. 

The xproj and yproj arrays, each of which includes a number of values equal to the number of pixels on 
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the boundary of the character from which features are to be extracted, are operated upon by the FSAMP 
subroutine as indicated in block 14 of Rg. 1, which "resamples" the xproj and yproj arrays to reduce (or 
Increase) the number of points from nperim to 64, and thereby produce the xsamp(0-63) anray and ysamp- 
(0-63) array. In most cases, the 64 resampled points represent a very substantial reduction from the original 

5 number of perimeter pixels without a substantial loss in accuracy of the extracted features. A dual complex 
Fourier transform subroutine of Rg. 9 then operates on the xsamp and ysamp anrays together to produce 
two sets of eight harmonic coefficients. Each of these harmonic coefficients then is nonmaiized in Fig, 10 by 
dividing it by the sums of the set of eight hannonic coefficients to which it belongs, and then taking its 
square root to thereby produce the size invariant, rotation invariant harmonic coefficient arrays ccoef(1-8) 

TO and dcoef(1-8). 

The prad(ip) and ptan(ip) anrays produced by DOPERIM each are operated upon by the "parametric 
resampling" subroutine PFSAMP. as indicated in block 24 of Rg. 1, using plen(ip) to produce the rsamp(0- 
63) array and tsamp(0-63) array, each of which has 64 points. The PFSAMP subroutine also produces the 
tnorm(0-63) an^ay when operating on the ptan(ip) array. The real Fourier transform subroutine of Fig. 16 

75 operates upon the rsamp and tnorm arrays to produce rcoef(1-8) and tcoef(l-8) normalized harmonic 
coefficients, which are extracted features. Also, the autocorrelation subroutine of Rg. 17 operates on these 
scanned arays to produce the rcor(1-63) and tcor(1-63) normalized autocorrelations, which are extracted 
features. Also, the rsamp array is operated upon by the MOMENT and SORT subroutines, the outputs of 
which are then scaled to produce extracted features as indicated in Rg. 1A. 

20 The tsamp(0-63) an-ay is operated upon by the DERIV subroutine of Rg..12, as indicated in block 22. to 
produce the tdiff(0-63) array. This array is operated upon by the HILOSUM subroutine of Rg. 13. as 
indicated in block 23 of Rg. 1, to produce thisum and tlosum, which are indicative of prominent convexities 
and concavities of the character to be recognized. This array also is operated on by the MOMENT and 
SORT subroutines, the outputs of which are scaled to produce further extracted features as indicated in Rg, 

25 1A. 

The DORAD subroutine of Rg. 15 designated by block 30 in Rg. 1. operates on P(x.y) and npix and 
prmax to produce the ring(0-7) array and slice (0-23) array. The ring(0-7) anray is itself an extracted feature. 
The real Fourier transform subroutine of Rg. 16 operates on the slice anray to produce the scoef(1-6) 
normalized harmonic coefficients which are extracted features. Also, the autocorrelation subroutine of Rg. 

30 17 operates on the slice an^ay to produce the scor(1-23) normalized autocon-elations which are extracted 
features. The ring and slice arrays are operated on by the MOMENT and SORT subroutines to produce 
extracted features as indicated in Rg. 1A. 

The DOHOLE subroutine of Rg. 20. designated by reference numeral 33 in Rg. 1. operates upon the 
HX(lh). HY(ih), nhole. ex. cy, and prmax variables to produce the hdist extracted feature, and simple ratios 

35 and taken between ahole, bhole and perim to compute the hrati and hrat2 extracted features. 

Next, the individual subroutines will be described. Refennng to Rg. 3, the DOCON subroutine produces 
various characteristics of six hypothetical minimum bounding rectangles of the character, each rotated 1 5 
degrees relative to the prior one. The minimum bounding rectangles, as typified b)y 32-0 and 32-1 shown in 
Rg. 2A. are not actually "drawn", but visualizing them can be helpful in understanding the resulting 

40 computed variables. The DOCON subroutine comprises Rgs. 3-6. Basically, the purpose of Rg. 3 is to find 
all the perimeter pixels of a character such as 25 of Rg. 2A that are likely candidates for the set of four 
touching perimeter points of any minimum bounding rectangle, by eliminating from consideration any linear 
and concave perimeter points. The purpose of Rg. 4 is to find the set of four touching perimeter points of 
each minimum bounding rectangle, and the purpose of Rg. 5 is to find the center point, height, width, area. 

4s and aspect ratio of each minimum bounding rectangle, as well as the average center point. Finally. Rg. 6 
computes the distances between each pair of center points of the minimum bounding rectangles and the 
length of the perimeter of the polygon formed by the center points. 

The DOCON subroutine starts at block 50 and initializes a previous direction variable pd to the value 
PD(nperim-l) which is the direction from the pixel previous to the upper leftmost pixel of the character. For 

50 example, in Rg. 2A. numeral 23A is the upper leftmost pixel of the character, and point 23 B is the location 
of the previous (nperim-l)th pixel. 

The subroutine then goes to block 51 and initializes the perimeter index ip to 0 and also sets an index c 
of a cpi an-ay to 0. The cpi anray is referred to as the convex pixel index anray. The subroutine then goes to 
block 52 and sets a current direction variable cd to PD(ip). 

55 It should be appreciated that the perimeter pixel arrays PX(ip) and PY(ip) contain all of the perimeter x 
and y coordinates. Fig. 3 provides a way of filling a new array, wherein the convex pixel index an-ay or c::;- 
(c) array can be used instead of ip as the index of PX and PY perimeter pixel arrays in order to point to 
pixels which are suitable candidates for touching any minimum bounding rectangle. 
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The subroutine of Rg. 3 goes from block 52 to decision block 53 and determines .f (cd-pd) .s less than 
or equal to -4. If this determination is affirmative, it means that the perimeter of the character has been 
tracked through a sufficient cumulative angle to pass through direction 0 one or. more times, ^aking .t 
necessary to increment the current direction cd by 8 as shown in block 54 to rnamtain concavity, tt^e 
determination of block 53 is negative, the subroutine goes to decision block 55 and determines .^^ the 
current direction (cd-pd) is greater than 4. If this determination is affirmative, it means that the perimeter of 
the character has been tracked through a sufficient cumulative positive angle to pass through direction 0 
one or more times, making it necessary to decrement the current direction cd by 8 as shown in block 56 o 
maintain convexity. If the determination of block 55 is negative, the program goes to decision block 57 o 
determine if the current direction cd is greater than the previous direction pd. If ^^^'^ determination is 
affirmative, it means Uiat the present portion of the perimeter is convex, rather than being '"f ^ 1=°"";^ ' 
as would be indicated by a negative determination of block 57. If the present portion o the pennneter is 
convex, the convex pixel index array is set to ip. the number of the current perimeter pixel in block 58. me 

cpi index c is incremented by 1 in block 59. ■ . , . r.arimotor 

The subroutine Uien goes to decision block 60 and determines if the present.pixel .s the last penmeter 
pixel of the character. This corresponds to a negative determination of decision block 60. If tiie oop is no 
completed, the subroutine goes from block 60 to block 61 and sets tine previous direction pd to the current 
direction cd. then goes to block 62 and increments the pixel index ip by 1. and returns » J^^f f J f^'^ 
continues. In block 63. the subroutine sets a convex pixel count cpnum to c. and goes to the subroutine c 

In F^'^ Iht first step is to initialize a rotation index i to 0. as indicated in block 70. The rotation index 
keeps track of tiie six different rotations of the six minimum bounding rectangles men^oned above, or as 
implemented, to keep track of the six 15 degree rotations of the perimeter of character 25 and the resulting 
sets of minimum bounding rectangle measurements, in block 71. the subroutine initializes the x and y 
25 coordinate minima xmin and ymin to a very positive value and initializes the x and yj=°°;d'"3j '"^'^^ 
xmax and ymax to a very negative value. The subroutine then goes to block 172 and sets a 90 degree 
rotation index j to i + 6. and sets an angle 0 to (iV/2)/6. where 3.14 and is used to designate 

multiplication. „ ^ i tca c^^te 

The subroutine ttien goes to block 73 and reinitializes the cpi index c to 0. goes to block 73A and sets 
30 the current x coordinate to x = PX(cpi(c)) and sets the current y coordinate to y = PY(cpi(c)). and ttien 

^°^ThL purpose of providing two rotation indexes i and j is so that for every rotation, two minimum 
bounding rectangle lengtiis. i.e.. a minimum bounding rectangle vertical length or height and a minimum 
bounding rectangle horizontal lengtii or width will be stored in clen(i) and clen(j> elements, respectively of 
35 the lengtii array. (Also see block 82 infra). 

In block 72. rotation index i is incremented, and fl also is incremented by 15 degrees. 

Blocks 73A, 74 75. 76, and 77 form a loop which examines every candidate convex point on th- 
perimeter of tiie character after incrementing 9 . in order to determine the minimum and maximum values o. 
the rotated x coordinate and ttie rotated y coordinate. In block 74. the subroutine sets tiie rotated x 

40 coordinate xr to 

xr = x'cos(fl) - y'sin(9) 
and sets the rotated y coordinate yr to yr = x-sin(e) + y*cos(e). 

The two foregoing equations serve to effectively rotate the point x.y counter-clockwise around the ongin 

4S °'° Then"thrsubroutine goes to block 75 and determines the extrema of the rotated object by setting xmin 
to the smaller of xr and the previous value of xmin. sets ymin to the smaller of yr and the previous value of 
ymin. sets xmax to the larger of xr and Uie previous value of xmax and sets ymax equal to the larger of yr 
and the previous value of ymax. The program then goes to block 76 determines if the convex pixel index 
array index c is less than the convex pixel count, cpnum-1. If this determination is affirmative, then there are 

5(7 more candidate convex perimeter pixels, so the subroutine goes to block 77. increments c by 1 and returns 
to block 73A. Otherwise, the subroutine goes via label C to Rg. 5. 

In block 80 of Fig. 5. the subroutine sets a rotated center point coordinate xrc to (xmax + xmin)/2 ana a 
rotated center point coordinate yrc to (ymax + ymin)/2. The subroutine then goes to block 81 and sets 
cxcp(i). the X coordinate of the center point of the minimum bounding rectangle for rotation i. to 

55 xrc'cos(e) + yrc'sin(fl), 

and sets cycp(i). the y coordinate of the center point of the present minimum bounding rectangle to 

-xrc*sin(e) + yrc'C0Sin(9). • • n r> ♦« »ho 

The two foregoing equations serve to effectively rotate the point xrc.yrc clockwise about the ongin o.o to me 
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point cxcp(i). cycp(i). 

The subroutine then goes to block 82 and sets width clen(i) to (xmax - xmin) and sets height clenO) to 
(ymax - ymin). The subroutine then, in block 83 sets cprd(i) to clen(i)*clenG). 

The subroutine then goes to block 84 and determines if the width clen(i) is less than the height clenG) 
5 for the present minimum bounding rectangle. If this determination is affimnative, the subroutine goes to 
block 85 and computes an aspect ratio cdiv(i) equal to the ratio of clen(i)/clen(j). Othenft^lse. the subroutine 
goes to block 88 and sets the aspect ratio cdiv(i) to the opposite ratio clen{i)/clen(i) so that the aspect ratio 
will always be less than or equal to 1 . 

In either case, the subroutine goes to block 87 to determine if the rotation index i is less than 5. If it is. 
70 the program goes to block 86. increments i. and returns via label B to block 71 of Rg. 4 to repeat the 
foregoing process for another 15 degree rotation. Otherwise, the subroutine goes to block 89 and sets cx to 
the average of the six cxcp(i) and sets cy equal to the average of the six cycp(i). The subroutine then goes 
via label D to Rg. 8. 

At this point, the subroutine has completed rotating character 25 to produce six minimum bounding 
75 rectangles. The variables pertaining to the six resulting center points are computed next. The first step, in 
block 90 of Rg. 6, is to initialize a first center point index i1 to 0 and to initialize a cdist index d to 0. The 
cdist index d takes on 15 values from 0 to 14. which is the number of lines tiiat can be drawn t>etween all 
pairs of the six center points of the rotated minimum bounding rectangles. Next, the subroutine goes to 
block 91 and initializes a second center point index 12 to i1 + 1. The subroutine then goes to block 92 and 
20 sets cdist(d) to 



The subroutine tiien goes to block 93 and determines if the second center point index i2 is less ttian 5. and 
if it is, it increments i2 and d, and returns to block 92 and sets a new value for cdist(d). When a negative 
detenmination is obtained from block 3. the subroutine goes to block 94 and determines if the first center 
point index i1 is less than 4. and if it is, increments i1 and returns to block 91. When a negative 
determination is obtained from block 95, the subroutine goes to block 97. In effect, the two loops fed by 
decision block 93 and 94 serve to measure distances from the first center point cx.cy to second, third, 
fourth and fiftii center points, and then measure tine distances from \he second center point to the tiiird. 
fourth, and fifth points, etc.. until distances between all possible pairs of the six center points have been 
measured to thereby produce the 15 values of cdist(0-14). 

In block 97. the subroutine reinitializes the first center point index i1 to 0 and also initializes the variable 
cdsum to 0. The subroutine then goes to block 98 and sets the second center point index i2 to (\^+^) 
(modS). The subroutine then goes to block 99 and increments cdsum, the sum of all of the distances 
between adjacent bounding rectangle center points by the amount 

f , — — ^ 

40 cdsum = \/(cxcp(i>-cxcp( j) )* + (cycp(i)-cycp( j) 

and then goes to block 100 and determines If i1 is less than 5. If it is. the subroutine increments i1 and 
retums to block 98, and otherwise returns. 

45 Referring next to Rg. 7, the DOPERIM subroutine operates on the same perimeter information as 
DOCON, namely nperim. perim, the PX(ip). PY{ip), and PD(ip) arrays, as well as cx.cy. previously computed 
by DOCON. In block 105 the previous direction variable pd is initialized to 0 and the start direction variable 
sd is initialized to PD(0) (pd and sd are only used in generating the ptan(ip) array). The subroutine then 
goes to block 106 and initializes the perimeter index ip to 0. and then goes to block 107 and sets xproj(ip) 

50 to PX(ip) and sets yproj(ip) to PY(ip). (The reason for the foregoing steps is to allow conversion from integer 
to floating point representation). Then, in block 108 the DOPERIfVl subroutine computes the length of the 
radius from the center cx.cy and sets prad(ip) equal to that radius, i.e., to 



The subroutine then goes to decision block 109 and determines if the direction PD(ip) is even or odd. If 
it is even, the subroutine goes to block 110. and sets plen(ip) to 1, since the present direction is either 




55 



prad (ip) 
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horizontal or vertical. If the present direction is not even, then the present direction is diagonal, and the 
subroutine sets plen(ip) equal to the square root of 2. In either case, the subroutine goes to block 112 and 
sets the current direction variable cd to so that the first direction is always 0 and the bias or start direction 
is subtracted from subsequent directions. 

5 The subroutine then goes to block 113. Blocks 113-116 of Fig. 7 accomplish exactly the same thing as 
blocks 53-56 of Rg. 3. In block 117 the subroutine sets ptan(ip) to the current direction cd. and sets the 
previous direction pd to the present value of cd. and then goes to decision block 118, which determines if 
the foregoing procedure has been performed for all of the perimeter points up to the nperim-l perimeter 
point. If this determination is negative, that is, if further perimeter points remain, the subroutine goes to 

70 block 119, increments the perimeter index ip by 1. and returns to block 107. Otherwise the subroutine 
computes prmax to be the maximum of the radius lengths in prad(ip) and returns. 

Next, the feature extraction routine goes to the FSAMP subroutine of Rg. 8. The FSAMP subroutine 
operates upon arrays of nperim elements, which is the number of perimeter pixels of the character 25. The 
FSAMP subroutine "resamples" these arrays to represent them by 64 points, which has found to be 

75 adequate for good accuracy. This reduction in the number of perimeter points to, a constant, small number 
greatly reduces the computational time required to obtain Fourier coefficients, moments, etc. and allows 
convenient accessing of stored cosine and sine tables to compute the Fourier transform. (It should be noted 
that the same FSAMP subroutine is executed to operate on xproj(ip) and yproj(ip). although in Rg. 8 it is 
shown only using the xproj and xsamp variables. If the FSAMP subroutine operates on yproj. then th- 

20 variable ysamp is used.) First, the subroutine goes to block 125 and sets a resampling size variable tsize 
equal to nperim/64. Note that there are nperim perimeter pixel points, and it is desired to convert these to 
64 resampled points, so it is necessary to define the sampling size as equal to nperim divided by 64. 

Next the subroutine goes to block 126 and initializes a sample sum ssum and also initializes a sample 
■ increment tsum to 0. The sample increment tsum is used to count samples having the sample size tsize. 

25 and ssum represents the sum of those values. 

Next the subroutine goes to block 127 and initializes the xproj index i and the xsamp index j to 0. The 
xproj index i counts through the nperim perimeter points and the xsamp index j counts through the 64 
resampled points. 

The subroutine then goes to block 128 and increments tsum by 1 and increments ssum by xproj(i). 

30 Next the subroutine goes to decision block 129 and determines if tsum is greater than or equal to tsize. 
If this determination if affirmative, it means that i has crossed a boundary needed to form another xsamp 
sample. The subroutine therefore decrements tsum by tsize, in block 130. goes to block 131 and sets 
sample sum remainder srem to tsum times xprojO). The subroutine then goes to block 132 and sets xsamp- 
0) to (ssum-srem)/tsize. increments j by 1 in block 134 and returns to decision block 129. If the 

35 determination of decision block 129 is negative, the subroutine goes to decision block 135 and determines 
if the xproj index i is less than nperim-1. If this determination is negative, it means that all nperim perimeter 
points have been resampled. and the subroutine returns to the catling program, if the determination of b\oc^ 
135 is affirmative, the subroutine determines that more resampling is required, increments the xproj index , 
by 1. and returns to block 128. 

40 (It should be noted that if the yproj array is being operated on by FSAMP, then xproj is replaced by 
yproj in block 127 and xsamp is replaced by ysamp in block 127. xproj is replaced by yproj in block 131. 
and xsamp is replaced by ysamp in block 132.) 

Perhaps the operation of the FSAMP subroutine can be further understood with reference to Fig. 8A, in 
which reference numeral 139 represents the xproj(i) array, plotted versus the xproj index i. If the distance 

45 between vertical line 139B and vertical ordinate 139C is tsize. xsamp(l) is the area under the curve 
between tsize line 139B and the zero ordinate 139C. This is the quantity computed in block 132 of Rg. 8. 
This value is represented by the horizontal line 139D. which represents a single resampled one of the 
permitted 64 points corresponding to nperim/64 perimeter pixels. If it is determined in block 129 that tsum 
is greater than tsize. the remainder srem is represented by the area under the curve 1 39 between vertical 

50 lines 139A and 139B. vertical line 139A representing tsum in this case. The amount srem is included in the 
xsamp(2) value computed in block 132 on the next pass and indicated by the level 139F, which extends to 
vertical line 139E, which is equal to 2 times tsize. 

The above subroutine accurately "shrinks" the nperim perimeter pixels to a more manageable sample 
size of 64 if nperim exceeds 64. and "extends" the nperim pixels to produce 64 samples if nperim is less 

55 than 64. 

As indicated in Rg. 1. the xsamp and ysamp arrays together are operated upon by the dual complex 
Fourier transform subroutine of Rg. 9, which computes two sets of eight harmonic coefficients of the xsamp 
and ysamp arrays. The CDFOURIER subroutine of Rgs. 9 and 10 goes first to block 140 and initializes the 
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index c, which is the Index for the ccoef variable and the dcoef variable, to 1. The subroutine then goes to 
block 141 and initializes the xsamp and ysamp index i. the cosine sums xcos and ycos, and the sine sums 
xsin and ysin to 0. The subroutine then goes to block 142. which sets an angle 6 to i'c'2 /64, since 64 
points represent the traversing of the perimeter of the character from the first pixel to the nperinrt-1 pixel, or 
5 one cycle. 

Next the subroutine goes to block 143 and increments xcos by xsamp(ircos(e), increments xsin by 
xsamp(i)*sin(e), increments ycos by ysamp(i)'cos(e), and increments ysin by ysamp(i)'sin(5). 

The subroutine then goes to decision block 144 and determines if the xsamp and ysamp index i is less 
than 63, and if this determination is affirmative, the inner loop of the subroutine is unfinished, so i is 
TO incremented by 1 and the subroutine returns to block 142. If the determination of decision block 144 Is 
negative, the inner loop of the subroutine is finished, and the subroutine goes to block 146 and sets ccoef-. 
(c) to 

ccoef(c) = (xcos + ysin)2 + (ycos - xsin)2. 
and dcoef(c) to 
75 dcoef(c) = xcos2 + xsin^ + ycos^ + ysin2. 

The subroutine then goes to block 147 and determines if the ccoef and dcoef index c is less than 8. and if it 
is. increments c by 1 and returns to block 141. If c is not less than 8, the subroutine goes to block 148 of 
Rg. 10. 

The coefficients ccoef(1-8) and dcoef(1-8) computed above are "raw" coefficients in that they are 
20 dependent upon the size of the character. This is unsatisfactory because the size d ependence makes it 
impossible to distinguish a relatively high amplitude harmonic of a small character from a relatively low 
amplitude harmonic of a large character, because both will have the same magnitude. In the prior art. the 
approach to normalizing the harmonic coefficients has been to divide the coefficients by the magnitude of 
the first harmonic or by the sum of the magnitudes of all of the harmonics. The former approach, 
25 normalizing with the amplitude of the first harmonic only, has the disadvantage that the first harmonic itself 
is a useful feature, and generally it is undesirable to scale or normalize with useful feature because 
informafional content of that feature is lost. 

The main disadvantage of the latter approach is that the higher numbered Fourier coefficients will be 
abnonmaily high if the character is very noisy. 
30 Continuing with Rg. 10, the variables csum and dsum are set equal to the sums of the respective sets 
of eight raw coiefficients in block 148, and the ccoef and dcoef index c is initialized to 1^ Next, block 149 
sets ccoef(c) to 

35 ccoef (c) « y ccoef (c) /csum' 

and dcoef(c) to 

^ dcoef (c) « y dcoef (c) /dsum^ . 

Thus, the approach is to normalize each of the magnitude-squared raw coefficients by the sum of the set of 
eight harmonic coefficients to which it belongs and then take the square root. 

45 The PFSAMP parametric resampling subroutine of Rg. 11 is similar to the FSAMP subroutine of Rg. 8, 
except that the former operates on the prad(ip) and ptan(ip) subroutines using plen(ip). the elements of 
which are not equaily spaced, as is the case for the xproj(ip) and yproj{ip) arrays. In Rg. 11, blocks 150 and 
151 are identical to blocks 125 and 126, respectively, of Rg. 8. Block 152 is similar to block 127. except 
that in the former, I is the prad index and j is the rsamp index. In block 153 of Rg. 11, tsum is 

50 parametrically incremented by the length plen(i) and ssum is incremented by the area plen(i)"prad(i). Blocks 
154, 155. 158, 159, 160. 161, and 162 of Rg. 11 are identical to blocks 129, 130. 133, 134. 135. 137. and 
136 of Fig. 8, respectively. In blocks 156, srem is set to the area tsum'prad(i). and in block 157 rsamp(j) is 
computed to (ssum - srem)/tsize. 

Rg, 11A is the same as Rg. 8A. except that the prad(i) samples are unevenly spaced in accordance 

55 with plen(i). If the PFSAMP routine of Fig. 1 1 operates on ptan(ip) instead of prad(ip). then prad and rsamp- 
(ip) in block 152 are replaced by ptan(ip) and tsamp. respectively. Tsamp is, in essence, an array of scaled 
perimeter tangent angles. Furthermore, if PFSAMP operates on ptan(ip), block 163 is inserted between 
block$ 157 and 158, indicating that tnorm(j) is set to tsamp(j)-8V64. 
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Block 163 IS added in order to correct tsamp by removing increments of "circularity". To understand 
what this means, refer to F\g. 11B, in which tsamp is plotted against j. The diagonal line 166 represents 
tsamp if the character represented is a circle 166A. The curve 167 is a plot of tsamp if the character is the 
"figure 8" shape 1688. The function tsamp(j) is not periodic, i.e., it does not return to 0, so a Fourier 

5 transform of tsamp(j) is not possible. However, if the diagonal line 166 is subtracted, i.e.. the increment of 
"circularity" is removed by subtracting the quantity r8/64. to produce tnorm(ip), then a meaningful Fourier 
transform of tnorm(0-63) can be computed as indicated in Fig. 1 . 

Fig. 12 shows a subroutine DERIV that performs a smoothing, derivative operation on the tsamp(0-63) 
array to produce the tdiff(0-63) array, which represents increments in the resampled direction codes as 

70 differences between successive smoothed points of the tsamp array. As subsequently described, this 
information is useful in evaluating the convex and concave portions of the character from which features are 
being extracted. This is useful because the English alphanumeric character set is partially discernible by 
analysis of the magnitudes of a character's convexities and concavities. 

In Rg. 12. the first step is to initialize the smoothing sum variable ssum to the sum of the first eight 

rs tsamp anray elements, as indicated in block 190. In block 191. the subroutine initializes a low smooth index j 
to 0 and initializes a high smooth index k to 7. This in effect establishes an eight element "moving window" 
from which a moving average is computed. The subroutine then goes to block 192 and sets tdiff(j) to 
ssum/8. The subroutine then goes to block 193 and decrements ssum by tsampG). to thereby remove that 
term from the window. 

20 The subroutine then goes to block 194 and determines if the high smooth index k is less than 64. and if 
it is. goes to block 195 and increments ssum by tsamp(k) to thereby add that term to the window. If k is not 
less than 64, the subroutine goes to block 195A and instead increments ssum by tsamp(k-64) + 8. in order 
to "wrap around". Next, the subroutine goes to decision block 196 and determines if the low smooth index j 
is less than 63. If it is, the subroutine goes to block 197. increments j. returns to block 192 and computes a 

25 new value of tdiffQ), i.e.. a new value of the moving average tdiffO). If the determination of block 196 is that 
the smoothing loop is finished, and the subroutine goes to block 198 and initializes a tdiff index i to 1. and 
enters another loop at block 199 and resets tdiff(i) to (tdiff(i + 1) - tdiff (i)). thereby computing the direction 
code increment. The subroutine then goes to decision block 200 and determines if the tdiff index i is less 
than 62, and if it is. increments i and returns to block 199. When the first 63 points of the tdiff(i) array have 

30 been computed, the subroutine obtains a negative determination from block 200, goes to block 202, and 
computes the value of the final point of the tdiff(i) array, namely point tdiff(63) to 
(8+tdiff(0)- tdiff{63)). 

in order to wrap around, tdiff is, in essence, an array of perimeter tangent angle increments. The subroutine 
then returns- 

35 Next, the HILOSUM subroutine of Rg. 13, which analyzes the concavities of the character from which 
features are being extracted, is described. The subroutine operates on the tdiff(i) array that contains the 
smoothed differences of the perimeter direction codes from the tsamp array. Understanding of thF 
HILOSUM subroutine of Rg. 13 can be aided by referring to Rg. 14. in which tdiff(i) is plotted for a "figurb 
8" character 25, Waveform 213 represents tdiff(0-63). Two values of the variable tol. a tolerance or 

40 threshold, namely tdiff equal to -1 and + 1 also are shown. Points A, 8, C. D, E, and F are identified on 
curve 213 to identify convexities A, C. D. and E and concavities B and F of the character. The convexities 
appear as peaks and the concavities appear as valleys in the tdiff curve. As can be seen from curve 21 3 in 
Fig. 14, areas 21 4E above the higher tol value of +1 represent prominent convexities, whereas areas 21 4D 
below the lower tbi value of -1 represent prominent concavities. 

45 Multiple values of tol can be empirically selected to produce a desired level of information about the 
concavities and convexities of the character from which features are being extracted. 

The HILOSUM subroutine of Rg. 13 first goes to block 205 and initializes a low count variable Inum. a 
low sum variable Isum, a high count variable hnum, a high sum variable hsum and a tdiff index i, all to 0. 
The subroutine then goes to decision block 206 and determines if the current value of tdiff(l) is less than tol. 

50 If the determination of block 206 is affirmative, the subroutine goes to block 207 and increments Inum by 1 
and Isum by tdiff(i). and goes to decision block 209. if the determination of block 206 is negative, the 
subroutine goes to block 208, increments hnum by 1. increments hsum by tdiff(i), and goes to decision 
block 209. Block 209 determines if i is less than 63, and if it is, the subroutine goes to block 210 and 
increments i by 1 and returns to block 206. Otherwise the subroutine goes to block 211 and sets tlosum to 

55 tlosum = (torinum • lsum)/64, 
and thisum to 

Ihisum = (hsum - torhnum)/64, 
and returns. In essence, the scaled sum of the convex areas such as 21 4E in Rg. 14 are representative by 
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this sum and the scaled sum of the areas 21 4D representing concavities are represented by tlosum. 

Next, the OORAD subroutine of Rg. 15. which fonms rings and slices as previously described with 
reference to Rg. 2 c, shows how character pixels are added into the rings and slices. In block 216 the 
subroutine sets a ring scale factor variable rsize to 7.99..yprmax, thereby scaling any radii to a tnjncated 
5 integer between 0 and 7. The subroutine then goes to block 217 and sets a slice scale factor variable ssize 
to 23.99..72ir, in order to scale all angles to a truncated integer between 0 and 23. 

In block 218. the subroutine Initializes all of the ring(0-7) array to 0 and also initializes all of the slice(0- 
23) array to 0. 

In block 219 the subroutine initializes y to 0. goes to block 220 and initializes x to 0. 

10 The initialized ring and slice variables now can be thought of as mathematical integrals into which dark 
pixels overiayed by the ring or slice are summed as the DORAD subroutine is executed. 

The subroutine then goes to decision block 221 and determines if P(x.y) is equal to 1. i.e., whether the 
present pixel is dark. If this detenmination is affirmative, the subroutine goes to block 222 and translates the 
X and y coordinates with reference to the center point cx.cy designated by reference numeral 26 in Rg. 2A. 

IS by setting x equal to x-cx and setting y equal to y-cy. 

The subroutine then goes to block 223 and sets the ring index r equal to rsize times the square root of 
(x2 + y2), thereby setting the ring index to the distance between the particular character pixel and the 
character center, and multiplies by rsize to truncate r to an integer iDetween 0 and 7. The subroutine then 
goes to block 224 and sets the slice index s equal to the arctangent of (-y/x), thereby setting the slice index 

20 to the angle formed by the particular character pixel and the positive x axis drawn through the character 
center and multiplies by ssize to truncate s to an integer between 0 and 23. The subroutine then goes to 
block 225 and mcrements ring(r) and slice{s),and goes to decision block 226. and if x is less than mx. the 
width of the minimum bounding rectangle at rotation 0. the subroutine increments x and returns to decision 
block 221 . If the detenmination of block 226 is negative, the subroutine goes to decision block 228 and 

25 determines if y is less than my, the height of the minimum bounding rectangle at rotation 0. If this 
determination is affirmative, the subroutine increments y and returns to block 220. and othenvise goes to 
block 230 and scales ring{0-7) and slice(0-23) by dividing them by npix to make these variables between 0 
and 1 and size-invariant Thus, the character array P(x.y) is scanned, line-by-line, and dark pixels are added 
to or included in appropriate ones of the eight defined rings and the appropriate ones of the 24 defined 

30 slices. Each of the eight rings and 24 slices then contains a certain fraction of the total dark pixels in the 
character. 

As previously mentioned with reference to Rg. 1, the slice(0-23) anray is not rotation-invariant, so the 
RFOURIER subroutine of Rg. 16. the CORREL subroutine of Rg. 17, the MOMENT subroutine of Rg. 18. 
and the SORT subroutine of Rg. 19 operate on the slice(0-23) array to produce si ze invaria nt and rotatio n 
35 in yariant feat ures. 

It should be noted that at this point in the execution of the feature extraction program, all of the 
computed variable arrays have been produced. As previously indicated, the RFOURIER subroutine of Rg. 
16 and the CORREL subroutine of Rg. 17. and/or the MOMENT subroutine of Rg. 18 and the SORT 
subroutine of Rg. 19 have been utilized to compute additional features in the cases where the computed 

40 variables arrays as shown in Rg. 1 are followed by a ' or +, respectively. In Rgs. 16-19. "xyz" designates 
any one of these arrays of n elements xyz(0),..jcyz(n-1). All of the features produced by these subroutines 
of Rgs. 16. 17, 18, and 19. are. of course rotation invariant. Features produced by RFOURIER and CORREL 
are size invariant due to normalization performed within those subroutines. Size invariance for the variables 
computed by MOMENT and SORT is achieved by dividing by an appropriate scale factor as discussed 

46 hereinafter. 

Next, the real Fourier transform subroutine RFOURIER of Rg. 16 will be described. It is quite similar to 
the complex Fourier transform subroutine of Rgs. 9 and 10 except it operates on a single array xyz, one of 
clen. cprd. cdiv, rsamp. tnorm. and slice, instead of xsamp and ysamp. and produces a single coef anay, 
• one of cIcoef(1-4). cpcoef(1-2). cdcoef(1-2). rcoef(1-8). tcoef(1-8) and scoef(1-6), respectively, instead of 
so ccoef and dcoef. The technique of subroutine 16 for Fourier transformation is well-known in the art. and is 
included here only for completeness. Block 170 of Rg. 16 is similar to block 140. except c is the index of 
coef. Block 171 is the same as block 141. except that i is the xyz index, and the cosine and sine sums are 
rcos and rsin instead of xcos ycos, xsin and ysin. Block 172 of Rg. 16 is identical to block 142 of Rg. 9. 
The computations shown in block 173 consists of half of the computations shown in block 143. The 
55 computation in block 176 is simpler than the two computations shown in block 146. Rnally, the normaliza- 
tion shown in blocks 180-184 of Rg. 16 using rsum are the same computation as the two normalizations 
shown in blocks 148-149B of Rg. 10 using csum and dsum. 

Next, the autocorrelation subroutine of Rg. 17 will be described. The technique of subroutine 17 for 
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autocorrelation is well-known in the art. and is included here only for completeness. Referring to Fig. 17, the 
subroutine CORREL first initializes all possible correlations acor(0-(n-1 )) to 0. The subroutine then goes to 
block 236 and initializes a correlation index c to 0. and enters an outer loop at block 237 by initializing the 
xyz index i to 0. The subroutine then enters an inner loop at block 238 and sets a shifted xyz index j to 
(i + c)modn. The subroutine then goes to block 239 and increments acor(c) by xyz(i)'xyzO). if this loop is 
unfinished, as is the case if 1 is less than n-1 as determined by decision block 240. the subroutine goes to 
block 241* and increments i, and returns to block 238 to continue the inner loop, which continues to 
compute the correlation variable acor(c). When the loop is completed, the subroutine goes to decision block 
242 and determines if the congelation index c is less than n-1. and if it is. goes to block 243. increments c, 
and returns to block 237 to continue the outer loop. When the outer loop is complete, the subroutine goes 
to block 244 and normalizes the computed values of the array acor(l-n-l) by dividing each element by the 
value of acor(O). the non-shifted autocorrelation, or sum of the squares of the xyz array, and returns. 

At this stage. coef(1-8) are raw coefficients, as are ccoef(1-8) and dcoef(1-8) after block 147 in Fig. 9. 
The elements of coef(1-8) are scaled by blocks 180 through 184 of Fig. 16 in the same way as ccoef(l-a) 
5 and dcoef(1-8) are scaled by blocks 148 through 149B in Rg. 10. 

The one-dimensional moment subroutine MOfvlENT of Rg. 18 simply computes moments in accor- 
dance with well-known art, and also is included here only for completeness. In Rg. 18. the average of the 
array is computed in block 250 by dividing all of the elements in the array by the number of the elements in 
the array. In block 251 the variance, skew, and kurtosis are computed using conventional well-know 
D formulas. In Rg. 19. the single block 246 indicates that the array is sorted into elements in ascending order 
(or any other predetermined order that may be useful). One skilled in the art can easily provide a routine to 
sort an array. In principle, new features can be generated by the arithmetic combination of any sets of 
elements from the sorted anray. Empirically, isolating the minimum and maximum elements and taking their 
sum, difference, product, and/or ratio is useful. Also, reducing the array by summing fixed numbers of 
5 adjacent elements, and taking their sums, differences, products and/or ratios has been found to generate 
additional useful features. 

Some computed variables that result from the fwlOMENT and SORT subroutines need to be scaled by 
appropriate scaling factors to yiefd size and variant extracted features. Arrays cien, cdist. and rsamp contain 
lengths or distances, and are proportional to linear dimensions of the character. Scaling their moments and 
to sorted elements by such linear values as perim, the minimum, maximum or average of the clen and prad 
arrays, or the square roots of the minimum, maximum or average of the cprd array effectively removes this 
linear dependence. An-ay cprd contains areas proportional to squared linear dimensions of the character. 
Scaling its moments and sorted elements by such area values as perim^, the. squares of the minimum, 
maximum or average of the clen and prad arrays, or the minimum, maximum or average of the cprd array. 
3S effectively removes this linear squared dependence. The remaining arrays cdiv, tdiff. ring and slice are 
already size invariant, and no further scaling of their moments and sorted elements is necessary. 

In Rg. 20. the DOHOLE subroutine sets the variables hx and hy to the averages indicated by thr 
equations in block 266, and sets hdist, the distance between the centroid hx.hy of the largest hole and tht> 
character center, thereby computing the length divided by the maximum radius length prmax 38 indicated 
40 in Rg. 2D. The variables hrati and hrat2 are set equal to the ratios ahole/perim and bhole/perim. 
respectively, as indicated in block 268. 

Most of the above-described set of extracted features has been found to be more immune to "noise" 
produced by variations in the character structure than is the case for the above-described "structurar 
approach to character recognition. This improved noise immunity seems to occur because often it is difficult 
45 to ascertain whether a "noise feature" really is noise or is a true part of the character to be recognized. This 
problem causes great difficulty in setting various threshold levels that are required in conventional structural 
character recognition techniques. Noise may appear as new segments in the structural skeleton, making it 
necessary to greatly expand the grammars, rule bases, or. decision trees that are essential to the structural 
approach to character recognition. Furthermore, obtaining the skeleton via the well-known medial axis 
50. transformation is much more expensive computationally than extracting the heretofore described features. 

^ In the above-described feature extraction system, using the borders of the character to be recognized 

instead of its pixel mass has been useful and adequate in obtaining most of the features, except for 
computation of the ring and slice features, while making a tremendous reduction in the number of required 
computations. The resampling techniques producing fixed numbers of elements in the arrays of which 
S5 Fourier transforms are computed makes it easy to utilize a few sets of precomputed trigonometric tables to 
obtain sines and cosines. However, the rotationally invariant ring and slice features obtained are so valuable 
that they, in conjunction with the other border-based features, have resulted in a highly optimized set of 
extracted features that can be efficiently recognized by a conventional statistical character recognition 
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decision tree. Some of the features have not actually been tested, but are expected to be useful and are 
deemed to be within the scope of this invention. 

The "robustness" and "richness" of the above-described set of extracted features plus their overall 
ease of computation contributes greatly to the character recognition capability of the system described in 
Rg. 21 . A decision tree was created using the aforementioned method of Suen by clustering 3200 character 
samples using most of the above defined extracted features (the clustering program automatically chooses 
the most effective subset of features to differentiate the character classes represented) 98.3% of this 
training set and 97.3% of an additional untrained test set of characters were recognized conrectly by this 
tree. Integerizing the algorithms was made easy by the bound on the dynamic range of numbers being 
added and multiplied together. 

Claims 

1. A system for extracting size-invariant, orientation-invariant features of a character, the system 
comprising in combination: 

(a) means for scanning the character and means responsive to the scanning means for producing a 
plurality of x and y coordinates of the perimeter pixels of the scanned character; 

(b) means for operating on the x and y coordinates to determinate values of a preselected parameter 
of a plurality of minimum bounding polygons each oriented differently relative to the character, each 
minimum bounding polygon having a plurality of sides each of which touches at least one respective 
perimeter point of the character, respectively; and 

(c) means for operating on the values to eliminate size and orientation ininformation therefrom. 

2. The system of Claim 1 wherein each minimum bounding polygon is a minimum bounding rectangle 
and including means for determining the dimensions of two mutually perpendicular sides of each minimum 
bounding rectangle. 

3. The system of Claim 2 including means for computing the area and an aspect ratio of each minimum 
bounding rectangle. 

4. The system of Claim 2 including means for determining the coordinates of the center points of each 
of the minimum bounding rectangles, and computing distances between those center points. 

" SVThe system of Claim 4 including means for determining the sum of the computed distances between 
adjacent center points. 

6. The system of Claim 1 including means for effectively rotating the character relative to an x,y 
coordinate system to produce the different relative orientations. 

7. A system for extracting features of a character, the system comprising in combination: 

(a) means for generating x and y coordinates of each respective perimeter pixel; 

(b) means for operating on the x and y coordinates to generate perimeter distances between each 
pair of perimeter pixels; 

(c) means for operating on the x and y coordinates and the center of the character to produce the 
radial distances between the center and the respective perimeter pixels; 

(d) means for resampling the radial distances by using the perimeter distances to produce a fixed 
number of resampled radial distances each corresponding to an equal proportion of the total perimeter 
length; and 

(e) means for operating on the resampled radial distances to eliminate size and orientation infonma- 
tion therefrom, to thereby produce the extracted features. 

8. The system of Claim 7 including means for performing a real Fourier transform on the resampled 
radial distances. 

9. A system for extracting features of a character, the system comprising in combination: 

(a) means for electronically scanning the character to produce perimeter pixel data and means 
responsive to the perimeter pixel data for generating x and y coordinates of each perimeter pixel; 

(b) means for operating on the x and y coordinates to generate perimeter distances between each 
pair of perimeter pixels; 

(c) means for operating on the x and y coordinates to produce a measurement of an angle tangent to 
the perimeter at every perimeter pixel; and 

(d) means for resampling the tangent angles by using the perimeter distances to produce a fixed 
number of resampled tangent angles each conresponding to an equal proportion of the total perimeter 
length. 
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10. The system of Claim 9 including means for subtracting an increment of circularity from each 
resampled tangent angle to produce the fixed number of corrected resampied tangent angles. 

11. The system of Claim 10 including means for performing a real Fourier transform on the corrected 
resampled tangent angles to produce harmonic coefficients. 

5 12- The system of Claim 11 including means for generating smoothed tangent angle increments by 
producing a moving average difference of the corrected resampled tangent angles: 

means for comparing each of the tangent angle increments to a threshold and incrementing a concavity 
indicator if that tangent angle increment is less than the concavity threshold; and 

means for comparing each of the tangent angle increments to a threshold and incrementing a convexity 
70 indicator if that tangent angle increment is greater than the convexity threshold. 
13 . A system for extracting features of a character, the system comprising: 
(a) means for scanning the character to produce perimeter pixel data and means operating on the 
perimeter pixel data for generating x and y coordinates, respectively, of each of the corresponding 
perimeter pixels; 

.75 (b) means for resampling the x and y coordinates by operating thereon to produce a fixed number of 

resampled x and y coordinates numbers each representing an equal proportion of the total number of x and 
y coordinates, respectively; and 

(c) means for performing a complex Fourier transform on the resampled x and coordinates to 
produce harmonic coefficients. 

20 14. A system for extracting features of a character, the method comprising: 

(a) means for scanning the character to produce pixel data; 

(b) means for operating on the pixel data to generate a plurality of hypothetical contiguous ring 
regions surrounding the character, and means for producing a plurality of ring variables corresponding to 
the respective ring regions; 

25 (c) means for determining the number of dark pixels of the characters that lie within each ring region 

and setting the corresponding ring variable to that number; and 

(d) means for eliminating size information from ring variable. 

1 5. A system for extracting features of a character, the system comprising; 
(a) means for scanning the character to produce pixel data; 
30 (b) means for operating on the pixel data to generate a plurality of hypothetical contiguous slice 

regions of a circle surrounding the character, and means for producing a plurality of slice variables 
corresponding to the respective slice regions; 

(c) means for determining the number of dark pixels of the characters the lie within each slice region 
and setting its corresponding slice variable equal to that number; and 

35 (d) means for eliminating size information from each of the slice variables. 
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T 



Set cprd(i )=clen( i clen (j ) 



y-82 



YES 



t85 



cdiv(i)=clen(i)/clen(j)j 




(^86 

cdiv(i)=clen(j)/clen(i)| 



YES 



88 



\ Increment i By One | 




89 



Set cx? average of (cxcp(i}} 
cy = average of {cycp(i)) 



© 



C SORT ) 
i 



(255 



Sort xyz Into sxyz in 
Ascending Order 



T 



r 

® 

C MOMENT ^ 



250 



Set xyz„.,^ = l/n 2. xyz ( i ) 



-251 



Arithmetically Combine 
Selected Sets of 
Elements of sxyz 



T 



256 



C returFT) 







1 n-l 




Set 


xyzvar= \ 


fl/n2(xyz(i)- 
i=0 




Set 


'^y^kew^^ 


/ n*l 

/l/n2(xyz(i)- 
i=0 




Set 


'^^kurt 


/l/n:E(xyz(i) 
i=0 


,4 

-'^y^avg^ 



FhzibcdIS C return ) 



0 281 725 



Initialize First Center Point 
Index il and cdist Index d to Zero 








Initialize Second 
Index i2 to il + 1 


Center Point 



90 



^91 



Set cdist(d)= 



N/(cxcp(i!)-cxcp(i2))2+(cycp(ll)-cycp(l2))^ 

1 ' 



Increment \Z 
and d By One 




Reinitialize Center Point Index il 
and cdsum to Zero 



Set Center Point Index 12 to 
(il + 1) (Mod 6) 



f98 



Increment cdsum By 



r99 



\/(cxcp(i)-cxcp(j)^+(cycp(i)-cycp(j))^ 

r 



100 




Increment il 
By One 



not 



NO 



C RETURN ) 
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DOPERIM ^ 



Initialize Previous Direction ly^^^ 
pd to 0 a Start Direction 
sd to PD(0) 



Initialize ip to Zero 



r/oe 



Set xproj(ip) = PX (ip) 
a yproj(ip) = PY(ip) 



Set prod ( ip)- 

\/(PX(ip)-cx)2 + (PY(ip)-cy)2 



110 



Set plen(ip)=i 

\ 



YES 




(III 



Set plen(ip)=\/T 



J 



Set Current Direction cd = 
PD(ip)-sd 



Increment 
cd By 8 



r-iie 



Decrement 
cd By 8 




Set ptan(ip)-cd 8 
pd=cd 



Increment 
ip By One 




rllS rU8 
YES^^ «P< 
nperim-l 



NO T 



U9 



Set prmax='^9'^<P''°'^<^>) 



C RETURN ^ 
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C FSAMP ) 



Set Resampling Size 
tsize=nperim/S4 



T 



Initialize Sample Sum ssum S 
Sample Increment tsum to Zero 



-/25 



^/26 



T 



Initialize xproj index i 8 



xsomp Index 



to Zero 



Increment tsum By One S 
ssum By xproj ( i ) 




Decrement tsum By tsize 



I30 



Set Sample Sum Remainder l/"' 
srem to tsum ¥: xproj ( i ) f 



131 



T 



Set xsGmp( j ) to 
( ssum — srem ) / tsize 



T 



Reset ssum to srem 



T 



Increment j By One 



132 
133 
134 



■137 



Increment 
By One 




C RETURN ^ 
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C CDFOURIER ) 



Initiolize ccoef(c} 8 
dcoef(c) To One 



-f40 



initialize xsamp ( i ) 8 ysamp(i) 
Index i, cosine Sums xcos, 
ycos a Sine Sums xsin, 
ysin To Zero 



Set Angle 9 = 
i -)f c->( 2T1/64 



Increment x cos By xsamp( i ) 
¥: cos (6), 

xsin By xsamp( i )X sin (9), 
ycos By ysamp( i )¥: cos(9K 
ysin By ysamp( i )•)(• sin (9) 



145 



Increment 
i By One 



YES 




144 



146 



Set ccoef (c) = {xcos + ysln)^+{ycos-xsin)^ 

a 

dcoef (c) = (xcos^+ xsin^H- ycos^ + ysin^ ) 



I47A 



'2 





Increment 
c By One 


YES^ 








/47 



(TO FI6. 10) 



0 281 725 



M9B 



increment 
c By One 



( FROM FIG. 9 ) 
T 



9 



8 

Set csum= 2 ccoef(c) 
c=i 
8 

dsum = ^2| dcoef(c) 



y/43 



Initialize ccoef(c) adcoef(c) 
Index c To One 



Reset ccoef (c ) = 

\/ ccoef (c ) /csum a 
dcoef(c)= ^ 

^/dcoef (c)/dsum 



T 



YES. 



c<8 




/49A 



NO 



C RETURN ) 



■m 



C HILOSUM ;) 



Initialize Low Count Inum, 
Low Sum I sum, High Count 
hnum. High Sum hsum, a 
tdiff (i ) Index i To Zero 



Increment Inum By 
One 8 Isum 
By tdiff ( i ) 



Increment 
i By One 



J 



y-205 




Increment hnum By 
One- a 

hsum By tdiff (i ) 



1 



211 



Set tlosum = 

(tol -X" lnum-lsum)/64 
a thisum = 

( hsum -to I ¥: hnum)/64 

— ^ — 

C RETURN ) 
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Q PFSAMP) 

Set Resampling Size tsize= /"-^^^ 
perim /64 



Initialize Sample Sum 
ssum 8 Sample 
Increment tsum To Zero 



151 



initialize prod ( i ) Index 
i a rsamp ( j ) 
Index i To Zero 



152 



Increment tsum By plen(i ) 

a ssum By 

plen ( i ) prod ( i ) 




ri53 



Decrement tsum By tsize 



(155 

i 



Set Sample Sum Remainder 
srem To tsum ¥: prod (i ) 



Set rsamp (j ) To 
( ssum-srem ) / tsize 



fl57 



Reset ssum To srem 



increment j By One 



/:I59 



Increment 
i By One 




/60 



f'^^ C RETURN ) 



163 



Set tnorm ( j ) To 
tsamp (j )-8-J^j/64 



0 281 725 





JF:rcB^±±B 



0 281 725 



C DERIV ) 



Initialize Smoothing Sum ssun 

To 5 tsamp( i ) 

i=0 



Initialize Low Snrwoth Index 
j To Zero a 

High Smooth Index k To 7 



/9/ 



Set tdiff (j ) = ssum/8 

J. 



/92 



Decrement ssum 
tsamp (j ) 

"7~ 







increment ssum 
By tsamp (k) 





r/93 



'I95A 



YES 



{197 



Increment 
j By One 




>N0 , 


Increment ssum By 




tsamp (k-64)+8 



196 



r/98 



Initialize tdiff 
Index i To One 



Reset tdiff ( i ) To 
tdiff (i+D- tdiff ( i ) 



/99 



YES 



Increment 
i By One 



If' 



20/ 




200 



^202 



Reset tdiff (63) To 
8+ tdiff (O) -tdiff (63) 

( RETURN ) 
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C DORAD ) 



227) 



I ncrement 
X By One 



Set Ring Scde Factor Size 
To 799.../prmax 

> 



Set Slice Scale Factor 
ssize =23.99... /2 IT 



f2l6 
r2l7 



T 



Initialize ring (0-7) 8i 
slice (0-23) To Zero 



J' 



218 




Reset Coordinates w.r.t. 
Center Points 

X = X — cx 

y = y-cy 



T 



Set Ring Index 
r = I nt( y(x2 + y2 )^ rsize ) 



223 



T 



224 



Set Slice Index 

s= lnt(tan~'(-y/x)-)f ssize) 



T 



(225 



Increment ring (r ) and 
slice (s) 



YES 



230) 



Scale ring (0-7) a 
slice (0-23) By npix 




■226 



228 
YES 



229 ) 



Increment 
y By One 



C return""^ 
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Increment 
i By One 



C RFOUR IERJ 

T 



Initialize coef ( c) 
Index c To one 



Initialize xyz( i ) Index i 
a cosine & Sine Sums 
rcos a rsin To Zero 



Set Angle 9 To 
i ^ c ^2 Tt /64 



Increment rcos By 
xyz(! )-)^cos{e) a 
rsin By 

xyz (i )^ sin(0) 



1 



178 



Set coef (c) To 
rcos^ -h rsin^ 

T 



Increment 
c By One 




177 



Set rsum= 2 coef(c) 
^ c=l 

\ 



Initialize coef (c ) 
Index c To One 



Reset coef (c )= 



^ coef (c)/rsum 



:I84 



Increment 


YES 


c By One 






133 



C RETURN ) 



170 



171 



172 



173 




176 



180 



(181 



182 



0 281 725 



C CORREL ^ 

Initialize acor ( O-(n-l)) 
To Zero 



Initialize* Correla t i on I ndex 
c To Zero 



Initialize xyz Index 
i To Zero 



Set Shifted Slice Index 
j To ( i+ c ) mod n 



^236 



^237 



238 



24/^ 



Increment acor(c) By 
xyz ( i )^ xyz (j ) 



Increment 




YES 


i By One 










r243 




Increment 






YES 


c By One 








240 




242 



C RETURN ^ 



j-239 





NO 


Scale acor ( 1 - 


-n-l ) 


By acor(O) 









^244 



0 281 725 



Cdohole ) 



r266 







nhole 


Set 


hx = 


2 PX(ih)/nhole 






ih=l 






nhole 


a 


hy = 


5 PY(ih) /nhole 




ih=l 



(267 



Set hdist= \/(cx-hx)^+(cy-hy)^ /prmaTj 



Set hrati = ahole/perim 
hrat2= bhole/perim 

t 

C RETURN ) 




8A 



1 



Noise Filters 



T 



Runlength Encoders 



T 



Object Builder 



T 



Object Classifier 



r 

r 

i 



i J ' 

I Runlength Decoder \ 

P(x.y).etci J 

I Border Tracker \ 



■2A 

■2B 
3A 

3B 
4A 

■4B 



P X.PY.PD.etcl 
] Feature ExtroctoT" 



Statistical Decision 
Tree Character 
Recognizet^^ 



5 
6A 



Rule- Based Context 
Character Recogn izer 

r — 



L. 


File Formatter 
















IBM PC-AT BASED 
WORK STATION 



(6B 



J 
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© A system extracts a set of size invariant, rotation 
invariant features from pixel data of a character and 
sends the set of features to a statistical decision tree 
to effectuate automatic recognition of a character. 
The set of extracted features Includes a first group 
obtained by generating arrays representable as di- 
mensions and other geometric qualities of six mini- 
mum bounding rectangles rotated about the char- 
acter, including the distances between the center 
points of the rotated bounding rectangles. A second 
group of features is extracted by generating a group 
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group are "resampied" to generate corresponding 
64 point arrays. The resampled arrays of direction 
codes is smoothed and incremental direction codes 
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group of features is extracted by dividing a circle 
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and counting the number of character pixels con- 
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transforms, autocorrelations, moment calculations, 
and sorting operations are performed on many of the 
aforementioned arrays to yield further size and ori- 
entation invariant features. The distance between the 
centroid of the largest hole of the character and the 
center of the character is computed. The ratios of 
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